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DETAILED ACTION 
Information Disclosure Statement 

1 . The information disclosure statements (IDS) submitted on 12/03/2004has been 
received, entered into the record, and considered. The submission is in compliance 
with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is 
being considered by the examiner. 

Priority 

2. Acknowledgment is made of applicant's claim for foreign priority based on an 
application filed in Denmark PA 1999 00827 on 06/11/1999. It is noted, however, that 
applicant has not filed a certified copy of the Denmark PA 1999 00827 application as 
required by 35 U.S.C. 119(b). 

Claim Objections 

3. Claims 1 , 6, and 8 objected to because of the following informalities: The 
examiner suggests that applicant remove all reference numbers from the 
aforementioned claims. For example, in claim 1, "(11)", "(12)", and "(S14, S16, S18)" 
should be removed. Appropriate correction is required. 

Claims 2-5 are objected to for incorporating the deficiencies of claim 1 . 
Claim Rejections • 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
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invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1-2, and 6-8 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Hyon (U.S. PGPUB 2002/0077135) and in view of Baker (U.S. Patent 6,546,417). 

6. Regarding claim 1 , Hyon teaches a display device comprising: 

A) a first storage portion (11) for storing beforehand a character code for specifying 
each character in said text and character shape data corresponding to said character 
code in a correlated manner (Hyon, Paragraphs 41-42, Figure 4); 

B) a second storage portion (12) for storing an image code for specifying said 
registered image and registered image data corresponding to said image code in a 
correlated manner according to registration processing by a user (Hyon, Paragraphs 25 
and 44, Figure 2); 

C) a display output portion (13) for outputting said text and said registered image 
(Hyon, Paragraph 21, Figure 2); and 

D) a display control portion (10) for causing said display output portion to output 
corresponding said text and said registered image based on display data containing a 
series of said character code, text attribute data, and said image code (Hyon, 
Paragraphs 21 and 42); 

The examiner notes that Hyon teaches "a first storage portion (11) for storing 
beforehand a character code for specifying each character in said text and 
character shape data corresponding to said character code in a correlated 
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manner" as "the user can input a text including typical characters, special characters, 
or emoticons within the range of a transmittable SMS message, by changing input 
modes" (Paragraph 41). The examiner further notes that Hyon teaches "a second 
storage portion (12) for storing an image code for specifying said registered 
image and registered image data corresponding to said image code in a 
correlated manner according to registration processing by a user" as "storage 18 
stores oriental or occidental emoticons according to the cultural area of a user" 
(Paragraph 25). The examiner further notes that Hyon teaches "a display output 
portion (13) for outputting said text and said registered image" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21). The examiner 
further notes that Hyon teaches "a display control portion (10) for causing said 
display output portion to output corresponding said text and said registered 
image based on display data containing a series of said character code, text 
attribute data, and said image code" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 16" (Paragraph 21) and "After an intended text is completed, the 
user instructs the mobile terminal to transmit the SMS message including the 
emoticon...the mobile terminal transmits the stored message" (Paragraph 42). The 
examiner further notes that it is common knowledge that when text messages are 
received via a device, the entire message is displayed at once. 
Hyon does not explicitly teach: 
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E) wherein said display control portion has image transforming means (S14, S16, S18) 
for transforming said registered image to be displayed according to said text attribute 
data. 

Baker, however, teaches "wherein said display control portion has image 
transforming means (S14, S16, S18) for transforming said registered image to be 
displayed according to said text attribute data" as "in order to accommodate the use 
of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided. ..at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Regarding claim 2, Hyon does not explicitly teach a display device comprising: 

A) wherein said text attribute data contains size attribute data indicating a character 
size of the corresponding text (Baker, Column 8, lines 26-40); and 

B) said image transforming means scales up/down said registered image according to 
said size attribute data (Baker, Column 8, lines 26-40). 
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Baker, however, teaches "wherein said text attribute data contains size 
attribute data indicating a character size of the corresponding text" as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided. ..at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40) and "said 
image transforming means scales up/down said registered image according to 
said size attribute data" as "in order to accommodate the use of different size fonts in 
the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Regarding claim 6, Hyon teaches a method comprising: 
A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Hyon, Paragraphs 25 and 44, Figure 2); and 
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B) displaying (S19) said text and said registered image simultaneously based on said 
transformed registered image, and said text attribute data and the character code for 
specifying each character in said text, and character shape data corresponding to said 
character code stored beforehand in a correlated manner (Hyon, Paragraphs 21 and 

42). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as 

"storage 18 stores oriental or occidental emoticons according to the cultural area of a 
user" (Paragraph 25). The examiner further notes that Hyon teaches "displaying 
(S19) said text and said registered image simultaneously based on said 
transformed registered image, and said text attribute data and the character code 
for specifying each character in said text, and character shape data 
corresponding to said character code stored beforehand in a correlated manner" 
as "a display 16 outputs display data and text messages generated in the mobile 
terminal. An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 
21) and 'After an intended text is completed, the user instructs the mobile terminal to 
transmit the SMS message including the emoticon...the mobile terminal transmits the 
stored message" (Paragraph 42). The examiner further notes that it is common 
knowledge that when text messages are received via a device, the entire message is 
displayed at once. 

Hyon does not explicitly teach: 
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C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming (S14, S16, S18) said registered image to be 
displayed according to said text attribute data. 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming (S14, 
S16, S18) said registered image to be displayed according to said text attribute 
data" as "in order to accommodate the use of different size fonts in the mailbox display, 
means for scaling the size of the icon graphics are also provided. ..at least one image 
for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Regarding claim 7, Hyon teaches a viewer program comprising: 
A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Hyon, Paragraphs 25 and 44, Figure 2); and 



Application/Control Number: 10/516,881 Page 9 

Art Unit: 2168 

B) displaying said text and said registered image simultaneously based on said 
transformed registered image, and said text attribute data and the character code for 
specifying each character in said text, and character shape data corresponding to said 
character code stored beforehand in a correlated manner (Hyon, Paragraphs 21 and 
42). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as 

"storage 18 stores oriental or occidental emoticons according to the cultural area of a 
user" (Paragraph 25). The examiner further notes that Hyon teaches "displaying said 
text and said registered image simultaneously based on said transformed 
registered image, and said text attribute data and the character code for 
specifying each character in said text, and character shape data corresponding to 
said character code stored beforehand in a correlated manner" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21) and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emotjcon...the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. 
Hyon does not explicitly teach: 
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C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data. 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Regarding claim 8, Hyon teaches a computer readable recording medium 
comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Hyon, Paragraphs 25 and 44, Figure 2); and 
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B) displaying said text and said registered image simultaneously based on said 
transformed registered image, and said text attribute data and the character code for 
specifying each character in said text, and character shape data corresponding to said 
character code stored beforehand in a correlated manner (Hyon, Paragraphs 21 and 

42). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user" as 

"storage 18 stores oriental or occidental emoticons according to the cultural area of a 
user" (Paragraph 25). The examiner further notes that Hyon teaches "displaying said 
text and said registered image simultaneously based on said transformed 
registered image, and said text attribute data and the character code for 
specifying each character in said text, and character shape data corresponding to 
said character code stored beforehand in a correlated manner" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21) and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. 
Hyon does not explicitly teach: 



Application/Control Number: 10/516,881 Page 12 

Art Unit: 2168 

C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data. 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided. ..at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

7. Claims 3-5 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hyon (U.S. PGPUB 2002/0077135) and in view of Baker (U.S. Patent 6,546,417) as 
applied to claims 1-2, and 6-8 and further in view of Ostermann et al. (U.S. Patent 
6,990,452). 
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8. Regarding claim 3, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains color attribute data indicating at least a fore 
color of a corresponding text; and 

B) said image transforming means converts a color of said registered image according 
to said color attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
color attribute data indicating at least a fore color of a corresponding text" as 

"The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color" (Column 11, lines 
43-50) and "said image transforming means converts a color of said registered 
image according to said color attribute data" as "The sender can associate typed 
words with an emoticon by underlining, coloring, highlighting, or by any other means. 
For example, the method may comprise providing the sender an option to assign a color 
to the at least typed one word such that the chosen emoticon begins to be presented by 
the animated entity to the recipient at the first typed word with the assigned color and 
the chosen emoticon presentation by the animated entity ends at the last typed word 
with the assigned color" (Column 11, lines 43-50). 
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It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 4, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said registered is a gray image, said image transforming means converts 
each pixel into a color made by mixing the fore color and a back color of said text at a 
ratio according to a pixel value. 

Ostermann, however, teaches "wherein said registered is a gray image, said 
image transforming means converts each pixel into a color made by mixing the 
fore color and a back color of said text at a ratio according to a pixel value" as 
"The increased intensity of the emoticon may be accomplished by changing the icon 
from black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
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the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 5, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains decoration attribute data indicating a type of 
a decoration applied to a corresponding text; and 

B) said image transforming means decorates said registered image according to said 
decoration attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
decoration attribute data indicating a type of a decoration applied to a 
corresponding text" as "The sender can associate typed words with an emoticon by 
underlining, coloring, highlighting, or by any other means... In this case, in a sentence 
such as "Hi John. :-) are vou pleased that the stock market is up ?" the underlining 
represents the highlighting wherein the sender chooses to begin the smile at the 
beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
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chosen duration of the emoticon" (Column 11, lines 43-61) and "said image 
transforming means decorates said registered image according to said 
decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up ?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon" (Column 11, lines 43-61). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Conclusion 

9. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. Patent 6,987,991 by Nelson on 17 January 2006. The subject matter 
disclosed therein is pertinent to that of claims 1-8 (e.g., methods automatically scale 
graphics into a text sentence that is sent to another user). 
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U.S. PGPUB 2002/0120653 by Kraft et al. on 29 August 2002. The subject 
matter disclosed therein is pertinent to that of claims 1-8 (e.g., methods automatically 
scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,584,479 by Chang et al. on 24 June 2003. The subject matter 
disclosed therein is pertinent to that of claims 1-8 (e.g., methods automatically scale 
graphics into a text sentence that is sent to another user). , 

U.S. Patent 6,456,305 by Qureshi etal. on 24 September 2002. The subject 
matter disclosed therein is pertinent to that of claims 1-8 (e.g., methods automatically 
scale graphics into a text sentence that is sent to another user). 

Contact Information 

1 0. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272- 
2731 . The examiner can normally be reached on Monday to Friday 8:20 am - 4:40 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Tim Vo can be reached (571) 272-3642. The fax number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov . Should 
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you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
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